Skip to content

v7 to v9 config migration script#669

Open
so-schen wants to merge 46 commits intomasterfrom
sc/upgrade-test
Open

v7 to v9 config migration script#669
so-schen wants to merge 46 commits intomasterfrom
sc/upgrade-test

Conversation

@so-schen
Copy link

@so-schen so-schen commented May 22, 2025 •
edited
Loading

Requirements

  1. We need devops team to package this migrate-config python script to rpc_node and supra version 9 docker image for mainnet, so that operator do not need to install locally @darpan-supraoracles

  2. We need to have snapshot bucket created for mainnet rpc and validator

        if is_validator; then
            bucket_name="mainnet-validator-snapshot"
        elif is_rpc; then
            bucket_name="mainnet-snapshot"

Note that aws access key may need to be updated in script

  1. This script support --assume-yes option to apply default migration values without prompting, it is most common use case for regular nodes, so that it is possible to batch processing config migration for our foundation nodes.

IMPORTANT note for DEVOPS
But for specially denoted nodes (i.e. archives, snapshot uploaders), use interactive prompt:
Do not use --assume-yes for these nodes, because the enable_snapshots and enable_pruning should retain original value instead of using the default.
Run the script in interactive prompt and confirm each update, and also check final output of migration .

  1. Operator should run interactively and follow the guide from the prompts. Operator should not run db migration, instead they should use snapshot sync to restart a node using v9 image

v7-v9 config migration docs

Install the config migration script

from root dir of repo, you can install it like below
pip install node_management/config_migration

Usage

$ migrate-config --help
Usage: migrate-config [OPTIONS] COMMAND [ARGS]...

  Migration CLI for Supra configs.

Options:
  --help  Show this message and exit.

Commands:
  rpc  Migrate RPC config.
  smr  Migrate SMR config.


$ migrate-config rpc  --help
Usage: migrate-config rpc [OPTIONS]

  Migrate RPC config.

Options:
  -p, --migrate-path [v7-v9]  Migration path (choices: v7-v9)  [required]
  -f, --from-file PATH        Source config file  [required]
  -t, --to-file PATH          Output config file  [required]
  -y, --assume-yes            Assume yes for all prompts (default: False)
  --help                      Show this message and exit.
  
$ migrate-config smr  --help
Usage: migrate-config smr [OPTIONS]

  Migrate SMR config.

Options:
  -p, --migrate-path [v7-v9]  Migration path (choices: v7-v9)  [required]
  -f, --from-file PATH        Source config file  [required]
  -t, --to-file PATH          Output config file  [required]
  -y, --assume-yes            Assume yes for all prompts (default: False)
  --help                      Show this message and exit.

example output

Running migration function: migrate_v7_to_v9

Scanning node root configuration ...
`connection_refresh_timeout_sec = 2` is not recommended for new version.
Do you want to apply the recommended config: `connection_refresh_timeout_sec = 1`? (assuming yes)
✓ Apply recommended config: `connection_refresh_timeout_sec = 1`

Scanning ledger configuration ...
✓ `enable_pruning` not found in original config, using new version's default value: True
✓ `enable_snapshots` not found in original config, using new version's default value: False

Scanning chain store configuration ...
✓ `enable_snapshots` not found in original config, using new version's default value: False

Scanning prune configuration ...

Scanning mempool configuration ...
`max_batch_delay_ms = 1500` is not recommended for new version.
Do you want to apply the recommended config: `max_batch_delay_ms = 500`? (assuming yes)
✓ Apply recommended config: `max_batch_delay_ms = 500`

Scanning moonshot configuration ...
`message_recency_bound_rounds = 20` is not recommended for new version.
Do you want to apply the recommended config: `message_recency_bound_rounds = 1000`? (assuming yes)
✓ Apply recommended config: `message_recency_bound_rounds = 1000`
`sync_retry_delay_ms = 2000` is not recommended for new version.
Do you want to apply the recommended config: `sync_retry_delay_ms = 1000`? (assuming yes)
✓ Apply recommended config: `sync_retry_delay_ms = 1000`
`timeout_delay_ms = 5000` is not recommended for new version.
Do you want to apply the recommended config: `timeout_delay_ms = 3500`? (assuming yes)
✓ Apply recommended config: `timeout_delay_ms = 3500`
Writing new config to /tmp/new_smr.toml
|----------------- Begin diff v7 to v9 -----------------|

..
|----------------- End diff v7 to v9 -----------------|

    ######################################################################
    # Config migrated from tests/smr_settings_v7.1.x.toml to /tmp/new_smr.toml.
    # 
    # Please review the diff above for changes made during migration.
    # 
    # Please ensure to use the new config file for target binary version.
    ######################################################################

E2e migration process from v7 to v9 example script

Note that below scripts are example, should be adapted and not directly used

  • see node_management/migrate_config_and_db_mainnet_v7_to_v9.sh is example to be used for our foundation node to migrate config and db
  • see node_management/migrate_config_v7_to_v9_docker.sh is example to be used for migrate config only for node using docker.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@isaacdoidge isaacdoidge isaacdoidge left review comments

@sjadiya-supra sjadiya-supra Awaiting requested review from sjadiya-supra

@vpanchal-supra vpanchal-supra Awaiting requested review from vpanchal-supra

@darpan-supraoracles darpan-supraoracles Awaiting requested review from darpan-supraoracles

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@so-schen @isaacdoidge